Causal Interpretations of Black-box Models

Abstract. Starting from the observation that Friedman’s partial dependence plot has exactly the same formula as Pearl’s back-door adjustment, we explore the possibility of extracting causal information from black-box models trained by machine learning algorithms. There are three requirements to make causal interpretations: a model with good predictive performance, some domain knowledge in the form of a causal diagram and suitable visualization tools. We provide several illustrative examples and find some interesting causal relations in these datasets.